Remote diagnosis of energy or resource-consuming devices based on usage data

ABSTRACT

Systems and methods are provided to retrieve or analyze usage data collected from a device or a facility where the device, optionally with devices are located, and identify useful features for making a diagnosis of the device. The diagnosis can be made before a system failure to reduce down time and inefficient use of the device, or after the system failure to expedite and facilitate diagnosis and repair. In addition to the usage data, such as energy and resource consumption, the system can also obtain information relating to the facility and the device&#39;s external environment which can be used for normalizing the usage data. Further, based on the diagnosis, the system can make suitable recommendations for repair, replacement, maintenance and upgrade.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.17/738,834, filed May 6, 2022, which is a continuation of U.S. patentapplication Ser. No. 16/042,850, filed Jul. 23, 2018, now U.S. Pat. No.11,366,465 B1, which claims the benefit under 35 U.S.C. § 119(e) of U.S.Provisional Application Ser. No. 62/536,411, filed Jul. 24, 2017, thecontent of which is incorporated by reference in its entirety into thepresent disclosure.

BACKGROUND

There is a growing number of devices in the home that are connected tothe Internet or computation-capable devices. The concept of “smart home”involves the control and automation of lighting, heating (such as smartthermostats), ventilation, air conditioning (HVAC), and security, aswell as home appliances such as washer/dryers, ovens orrefrigerators/freezers. These devices, when remotely monitored andcontrolled via the Internet, are an important constituent of theInternet of Things (IoT). Likewise, there are growing interests in smartoffices, smart factories and smart cities. In addition to providingconvenience in using and controlling these devices, data collected fromthese devices can be used for detecting problems, increasing efficiency,and saving energy, and thus are ultimately beneficial to the owners ofthe facilities and the society at large.

For example, smart thermostats can be used for controlling a facility'sheating and/or air conditioning. Smart thermostats are typicallyconnected to the internet and allow users to adjust heating settingsfrom other internet-connected devices, such as smartphones. This allowsusers to easily adjust the temperature remotely. Smart thermostats alsorecord internal/external temperatures, time the HVAC system that hasbeen running and can send notifications to a user. Another exampledevice for collecting data from a device is a smart meter that recordsconsumption of electric energy in intervals of an hour or less andcommunicates that information back to a central system for monitoringand billing.

The enormous amounts of data generated from these devices can present agreat challenge to users that attempt to identify useful informationfrom the data for the purpose of energy saving or maintenance.

SUMMARY

Typically, repair or replacement of an energy or resource-consumingdevice is triggered by a system failure. It can be readily appreciated,however, that the device may have already been malfunctioning for aperiod of time before the failure, or functioning at a lower efficiencyor generating a less-than-desired output. Some of the devices may comewith a limited set of diagnostic tools for detecting malfunctioning, butit is up to the user to decide on when to use the tools. As a result,even such limited tools are rarely used, leaving the devices operatingunder suboptimal conditions for a long time, which can waste energy andresources and cause unnecessary damages to the devices.

A claimed solution rooted in computer technology overcomes problemsspecifically arising in the realm of computer technology. In variousimplementations, a computing system is configured to retrieve or analyzedata collected from devices and/or the facility where the devices,optionally together with other devices, are located, and identifyinformation from the data that may be useful for diagnosis. In someimplementations, a computing system is configured to conduct a diagnosisfor one or more devices and report results of the diagnosis to a user.In some implementations, a computing system is configured to presentvarious types of data relating to the diagnosis on a suitable userinterface which enables a user to visualize, analyze and annotate thedata. In some implementations, a computing system is configured to makerecommendations, based on the diagnosis, with respect to repair,replacement, maintenance and upgrades.

In accordance with one embodiment of the present disclosure, provided isa method of determining the efficiency of a device at a facility. Insome embodiments, the method entails retrieving energy or resource usagedata for the device or the facility for a period of time; obtaining aproperty of the facility and environmental information during the periodof time; normalizing the usage data with the property of the facilityand the environmental information; and determining, by one or moreprocessors, the efficiency of the device with one or more featuresextracted from the normalized usage data with a data analysis model.

In some embodiments, the energy or resource usage comprises usage ofelectricity or natural gas. In some embodiments, the property of thefacility is selected from square footage of the facility, square footageof a room in which the device operates, age of the facility, type ofinsulation used in the facility, direction of windows of the facility,and the combination thereof. In some embodiments, the environmentalinformation comprises external temperature, humidity, strength ofsunlight, atmospheric pressure, altitude, latitude, wind speed and thecombinations thereof. In some embodiments, the normalization comprisesdetermination of expected energy or resource usage for the type of thedevice based on the property of the facility and the environmentalinformation.

In some embodiments, the one or more features is selected from the groupconsisting of external temperature gradient, one or more metrics ofrolling averages, maximal usage, distribution of usages, firstderivative of a usage curve, frequency and duration of on and off,heights of peaks, heights of troughs, differences between peaks andtroughs, differences between adjacent peaks and troughs, temperatures atwhich the device is switched on, temperatures at which the device isswitched off, and combinations thereof.

In some embodiments, the data analysis model is selected from randomforest, support vector machine, neural network, linear discriminantanalysis, quadratic discriminant analysis, and nearest neighbor. In someembodiments, the one or more features are selected by a methodcomprising calculating a receiver operator characteristics (ROC) curvewherein the area under curve (AUC) of the ROC curve is indicative of thediagnostic ability of a test feature.

In some embodiments, the method further entails displaying, on a userinterface, the energy or resource usage data over the period of time,plotting the efficiency over the period of the time on the userinterface, and/or displaying a recommendation, based on the determinedefficiency, for repair, maintenance, replacement or upgrade.

Systems and non-transitory computer-readable medium are also providedthat are suitably configured to implements the various embodiments ofthe above methods.

BRIEF DESCRIPTION OF THE DRAWINGS

Certain features of various embodiments of the present technology areset forth with particularity in the appended claims. A betterunderstanding of the features and advantages of the technology will beobtained by reference to the following detailed description that setsforth illustrative embodiments, in which the principles of thetechnology are utilized, and the accompanying drawings of which:

FIG. 1A shows example electricity usage data of a facility before (upperpanel) and after (lower panel) a repair.

FIG. 1B shows that the external temperature gradients can be useful fordistinguishing usage data obtained when a HVAC system at the facility isoperated under a relatively inefficient condition (e.g., before repair,left panel) from those obtained when the HVAC system is operated under arelatively efficient condition (e.g., after repair, right panel).

FIG. 2 show a few different receiver operator characteristics (ROC)curves which can be used to evaluate the value of a candidate featurefor assessing a device's efficiency.

FIG. 3 shows a plot of the raw usage data (upper panel) and a plot ofcalculated efficiency scores (lower panel).

FIG. 4 illustrates an example environment for generating diagnosisfeatures and models and using the features and models for makingdiagnosis of a device.

FIG. 5 illustrates an example process flow chart of a method, accordingto some implementations.

FIG. 6 illustrates a block diagram of an example computer system inwhich any of the implementations described herein may be implemented.

DETAILED DESCRIPTION

The technology described herein, in some embodiments, relates to systemsand methods for retrieving and analyzing usage data generated from theoperation of devices. The term “energy or resource-consuming devices” or“devices” as used herein refers to electrical, mechanical, or heatingdevices which accomplish one or more household, office, manufacturing,farming or other similar functions and require consumption of energy orother resources (e.g., natural gas). Non-limiting examples of devicesinclude lighting fixtures, heaters, ventilators, air conditioners,washers and dryers, ovens, stoves, toasters, refrigerators or freezers,home or office computers, home theater systems, printers, copier,projectors, and fans.

“Usage data” of a device or a facility wherein the device is located caninclude energy or resource consumption and operation conditions andresults. For example, the data may include temperature of certainportion of the device during operation which may be a result of theoperation or malfunctioning. Another example is the energy consumptionof the device. Energy consumption may be the use of electricity by thedevice or consumption of a different energy source, such as natural gas.The usage data may be obtained from the device directly if a suitablesensor or meter is equipped with the device. In addition oralternatively, the usage data of a device can be derived from theoverall usage of the facility wherein the device is located. In someexamples, the data can be represented as or constructed into a timeseries, such as energy (or resource) consumptions at a number ofconsecutive time intervals. In another example, the data may includemessages or signals sent from the device that include troubleshootingcodes. In another example, the data may include the make and model ofthe device, the manufacturing date, the date of first operation, and/orthe expected life span (or alternatively the expected date ofreplacement or maintenance).

In some embodiments, the computing system is also configured to retrieveinformation about the environment in which the device operates. Examplesof such information include temperature inside the facility, temperatureoutside the facility, humidity, strength of sunlight to which thefacility is exposed, atmospheric pressure, altitude, latitude, and windspeed. A particular example is the “external temperature,” which can bethe temperature outside the facility for a facility-wide device such ascentral air conditioner, or the temperature inside a facility where aregional device (e.g., freezer or refrigerator) is located. Theenvironmental information, in one embodiment, can be obtained from aweather database or obtained from meters installed in the facility.

In some embodiments, the computing system is further configured toreceive property information about the facility, such as the squarefootage of the facility or the room in which the device operates. Otherexamples of property information about the facility include the age ofthe facility, type of insulation used in the facility, direction ofwindows of the facility, the date of the most recent renovation, andenergy usage from another device in the facility, without limitation.The facility property information, in some embodiments, may be obtainedfrom a real estate database.

The usage data used for the analysis, in some embodiments, can bepre-processed. Pre-processing may include, without limitation, removalof obvious errors such as outliers, and normalization of a time seriessuch that the data can be compared to a different time series or adifferent device of the same kind. Normalization can be done with apredefined range, an internal reference, or an external reference.

In some embodiments, the normalization is done with reference toinformation about the environment and/or the facility, both of which aredescribed above. For instance, for an air conditioning (AC) unit, thereference may include the square footage of the facility, the outsidetemperature of the facility, and/or whether the AC unit is used for theentire facility or just one or more rooms in the facility. In someembodiments, the normalization uses a regression methodology to estimateexpected usage based on the environmental and facility information asindependent variables. In some embodiments, the normalized value isobtained by subtracting the calculated expected usage from the observedusage, or alternatively further adding the difference back to a meanusage.

The usage data can be analyzed with various methods including, withoutlimitation, those commonly used in pattern recognition and machinelearning. The analysis may be supervised or unsupervised, may beparametric or non-parametric. In a supervised analysis, for instance,the data can be categorized into two or more groups. In one embodiment,a goal of the data analysis may be to identify attributes (“features”)from the data series (“data sets”) that are useful for conducting adiagnosis of the device. Accordingly, the data can be categorized intoan “efficient” group and an “inefficient” group.

The terms “efficient” and “inefficient” are relevant terms to denotedifferent conditions under which a device is operated, whereas theefficient condition is a more-desired condition and the inefficientcondition is less desired or not desired. In one example, an efficientcondition is a condition more similar to when the device is new or hasjust been maintained or repaired, than to when the device breaks down orwhen a repair, replacement or other types of maintenance services arerequired.

In some embodiments, a group of data sets representing an efficientcondition can be obtained from a device shortly after a repair orreplacement. A group of data sets representing an inefficient condition,accordingly, can be obtained from the same device (or its replacement)prior to the repair or replacement. FIG. 1A includes the illustration ofa data set collected from a facility having a HVAC system device beforea repair (upper panel 101) that represents an inefficient condition. Adata set collected from the same facility after the repair of the HVACsystem (lower panel 103) represents an efficient condition of the HVACsystem. In each panel, the curve 102 is constructed with data points,each of which represents the amount of energy consumption for a15-minute time interval in the facility.

After the data sets are categorized into groups, a supervised learningmethod can be used to identify features useful for distinguishing datasets representing efficient conditions from those representinginefficient conditions. In some embodiments, candidate features can bechosen and evaluated.

An example candidate feature is an “external temperature gradient,”which denotes the additional energy consumption per degree change inexternal temperature. The idea of using external temperature gradientsto distinguish an efficient condition from an inefficient condition isillustrated in FIG. 1B. In FIG. 1B, each panel (e.g., 121) presents ascatter plot of quarter-hour energy consumption over externaltemperatures and includes a trend line, 122 and 123. Apparently, underthe inefficient condition (before repair), there are generally higherenergy consumption increase per degree increase in the externaltemperature. Accordingly, these charts show that the externaltemperature gradient can be a useful feature for diagnosis.

Another example candidate feature can be selected from various metricsof a rolling window. For a data set with data points collected every 15minutes, for example, a rolling window can be an hour, or two hours,without limitation. Transformation from the original data set to onewith the rolling window, therefore, can smoothen the curves and reduceerrors. Upon such transformation, certain metrics can be obtained, suchas maximum, minimum, medium, and distribution (e.g., at how manypercentage of the time the rolling average is below a threshold value)of the rolling averages.

In addition to external temperature gradient and the metrics based onrolling windows, other example candidate features include maximal usage(e.g., how often the data points fall in the general top decile),distribution of usage (e.g., percentages of data points at low,low-medium, high-medium and high regions), first derivative of the usagecurve, frequency and duration of on/off of the device, heights of peaks,heights of troughs, differences between peaks and troughs, differencesbetween adjacent peaks and troughs, temperatures at which the device isswitched on, temperatures at which the device is switched off, changesor trends of any of the features over time, and time from most recentrepair or replacement. To screen for new candidate features, a data setcan be divided into a training set and testing set where a data analysismodel for one or more features can be trained with the training set andtested with the testing set.

Candidate features can also be evaluated individually. For instance, areceiver operator characteristics (ROC) curve can be calculated for acandidate feature. Each ROC curve has an associated area under the curve(AUC). A feature that does not include useful information for diagnosiswould have an AUC of 0.5 An AUC of close to 1, by contrast, indicates avery good feature. These are illustrated in FIG. 2 which includes a ROCchart (201) with four different curves 202-205. The AUC of curve 202 (nofeatures are used) is about 0.5; the AUC of curve 203 is slightlygreater than 202 suggesting that the underlying feature is useful; theAUC of curve 204 is even greater indicating a strong underlying feature;and the AUC of curve 205 is almost 1, suggesting an almost-perfectfeature. Other pattern recognition and machine learning approaches canalso be used to screen for or evaluate candidate features, such asrandom forest, support vector machine, neural network, lineardiscriminant analysis, quadratic discriminant analysis, and nearestneighbor.

In some embodiments, two or more features are used in combination. Oneway of identifying such combinations uses a forward selection method. Aforward selection method, in one embodiment, starts with abest-performing feature among the candidate features and checks whichother feature, when used in combination with the first selected feature,performs the best (e.g., generates the highest AUC) among the differentcombinations. Once such a two-feature combination is obtained, a thirdfeature may be identified from the remaining candidate features in asimilar fashion. Other methods of identifying feature combinationsinclude backward elimination, exhaustive search of all combinations, andgenetic algorithms.

For any selected feature or feature set, a data analysis model can alsobe identified which, along with the feature or feature set, are able tocorrectly associate a data set to a related condition. In oneembodiment, the data analysis model is a random forest model. In anotherembodiment, the model is a support vector machine. In some embodiments,the model is selected from neural network, linear discriminant analysis,quadratic discriminant analysis, and nearest neighbor, withoutlimitation.

In some implementations, a computing system is configured to conduct adiagnosis for one or more devices with one or more selected features.The diagnosis, in some embodiments, generates an efficiency scoresuggesting how likely a device is operating under an efficient orinefficient condition. For instance, such a score may be an efficiencyscore in the range of 0 to 100, where 100 denotes a condition undermaximal efficiency and 0 being the opposite. For instance, in FIG. 3 ,based on the raw data shown as the curve 303 in chart 301, the systemuses a selected feature set and a corresponding model to generate anefficiency score(s) for each data set. Some example efficacy scores areshown in curves 304 in FIG. 3 , lower panel. It is noted that, in thisfigure, the efficiency scores are only calculated during periods whenthe device operates at relatively high intensity, for example, anintensity greater than a threshold intensity.

Comparing the efficiency scores in period A, period B and the first halfof period C, it is apparent that the relevant efficiency of the testeddevice goes down at each period. At time point 305, however, theefficiency score shoots up dramatically. Apparently, the user of thedevice becomes aware of the low efficiency of the device which likelycomes with other malfunctioning symptoms, and has the device repaired.

Rather than waiting for the device to break down as it may be the caseof FIG. 3 , it may be helpful to make the user aware of the decliningefficiency of the device so that a repair or maintenance can beconducted preemptively (e.g., during period B in FIG. 3 ), to avoid orminimize the inefficient use of energy during the first half of periodC. Further, in addition to helping the user save energy by alerting theuser to increasingly inefficient functioning, this technology can alsohelp the user identify “silent” problems before a total system failure.Even after a system failure, this technology can be greatly beneficial.For instance, the energy or resource provider or maintenance team can beimmediately notified so that appropriate action (e.g., ordering parts,shutting down the inefficient device) can be taken expediently. Inaccordance with one embodiment of the present disclosure, therefore,provided are computer systems and methods configured for displaying datacollected from the one or more devices, or the facility wherein thedevices are located, along with results of the data analysis andrecommendations. In one embodiment, the data are displayed on a userinterface which includes a panel using chart or other suitable graphicmeans to represent the usage data. In another embodiment, automaticalerts can be generated by the system from analyzing the data and thealerts can be displayed or sent to the user or an appropriatemaintenance team.

In one embodiment, the user interface includes a chart showing therelative efficiency of the device over a period of time. The showing ofthe efficiency data itself can be useful for a user, a technician, amaintenance team, or a vendor of services or devices, to decide onwhether a repair, replacement or maintenance is required. In someembodiments, nevertheless, the user interface further shows markings,tagging, text, or other means to make a recommendation for repair,replacement or maintenance for a user. In one embodiment, therecommendation includes replacement with a different make or model ofthe device and the recommendation can optionally include a chart orstatistic summary highlighting the improved performance of therecommended make or model. In some embodiments, the recommendationfurther includes an estimated saving (of energy, resource, or cost) ifthe recommendation is adopted.

A suitable recommendation, in one embodiment, entails alerting a user(e.g., the facility owner or tenant), a technician, a maintenance team,or a vendor of services or devices that the respective device is notoperating under an efficient condition. Along with the alert, therecommendation can further include a history record of the device suchas a decline of the efficacy over time. In some embodiments, therecommendation includes a comparison to a different device of the samekind which may suggest that an upgrade may be able to overcome certainlimitation of the current device. For instance, the comparison may showthat the current device has a steeper decline in the same period ascompared to a new model.

In some embodiments, the recommendation includes a calculated saving ofcost based on the recommendation. For example, with a recommendation fora repair or replacement, a number is presented to show that, upon suchrepair or replacement, a monthly saving of $100 can be achieved. In someembodiments, a follow-up is carried out which entails calculating theefficacy after a repair recommended by the system, to show that actualsavings have been achieved.

Computing Environments, Modules and Methods

FIG. 4 depicts an example environment 401 for facilitatingidentification of useful features and data analysis models for thediagnosis of devices and using the features and models to make diagnosisand recommendations. In one implementation, the environment 401 mayinclude one or more of a computer system 420, a user device 430, afeature data source 411, a data analysis model source 412, a facilityinformation data source 413, and an environmental information datasource 414, in communication via network 402, and/or other components.The data sources 411-413 are illustrated in FIG. 4 as being separatefrom the computer system 420 and the user device 430. In someimplementations, some or all of the data sources 411-414 may be storedon computer system 420, user device 430, and/or at a remote location. Insome implementations, the data sources 411-414 may be stored in the samelocation and/or may be stored in the same database. As illustrated inFIG. 4 , each of the software modules may be in operation on user device430 and/or on computer system 420. Various aspects of the system mayoperate on computer system 420 and/or on one or more user devices 430.That is, the various software modules described herein may each operateon one or both of computer system 420 and/or user device 430.

The data sources 411-414 may be computer memories configured to storedata. Further, the data sources 411-414 may store data formattedaccording to object based data structures as described above. In someembodiments, the feature data source 411 may store candidate features orselected features useful for diagnosis. In some embodiments, the dataanalysis model source 412 stores information about data analysis modelsand their associated features and related parameters. In someembodiments, the facility information data source 413 stores informationabout properties of the facility that may be used for normalization oranalysis of device usage data. In some embodiments, the environmentalinformation data source 414 stores information about the environment inwhich the device is operated.

The computer system 420 may be configured as a server (e.g., having oneor more server blades, processors, etc.), a personal computer (e.g., adesktop computer, a laptop computer, etc.), a smartphone, a tabletcomputing device, and/or other computing device that can be programmedto receive tabular data or object based data, provide services for themanipulation of the data, and provide services for transformation anddisplay of the data.

The computer system 420 may include one or more processors 422, one ormore storage devices 424, and/or other components. Processors 422 may beprogrammed by one or more computer program instructions stored onstorage device 424. For example, processors 422 may be programmed bydatabase access module 425, data normalization module 426, featureselection module 427, diagnosis module 428, recommendation module 429,and/or other instructions that program computer system 420 to performvarious operations, each of which are described in greater detailherein. As used herein, for convenience, the various instructionmodules, systems, and engines will be described as performing anoperation, when, in fact, the various instructions program theprocessors 422 (and therefore computer system 420) to perform theoperation. Further details and features of a computer system 420configured for implementing features of the described technology may beunderstood with respect to computer system 600 as illustrated in FIG. 6.

User device(s) 430 may be configured as a server (e.g., having one ormore server blades, processors, etc.), a personal computer (e.g., adesktop computer, a laptop computer, etc.), a smartphone, a tabletcomputing device, and/or other device that can be programmed to receivetabular data or object based data, provide services for the manipulationof the data, and provide services for transformation and display of thedata. In some embodiments, two or more user devices are included, eachof which is connected to one another or to the computer system 420 overa network.

User device 430 may include one or more processors 432, one or morestorage devices 424, and/or other components. Processors 424 may beprogrammed by one or more computer program instructions. For example,processors 422 may be programmed by database access module 425, deviceinformation module 430, visualization module 431, annotation module 432,comparison module 433, and/or other instructions that program computersystem 420 to perform various operations, each of which are described ingreater detail herein. As used herein, for convenience, the variousinstruction modules, systems, and engines will be described asperforming an operation, when, in fact, the various instructions programthe processors 422 (and therefore computer system 420) to perform theoperation.

In various implementations, the database access module 425 may be asoftware module operating on computer system 420 and/or user device 430.Database access module 425 may be configured to provide system access todata sources, e.g., the data sources 411-414. Database access module 425may be configured to read and write to data sources 411-414, as well ascarry out searches, queries, and any other database functionalityrequired by computer system 420 and/or user device 430.

The data normalization module 426, in some embodiments, is configured toretrieve usage data for a device or the facility where the device islocated and normalize the data with one or more properties of thefacility (from data source 413) and certain environmental information(from data 414). The feature selection module 427, in some embodiments,is configured to evaluate and select features from a list of candidatefeatures which may be used together with a data analysis model formaking a diagnosis, which may be conducted by the diagnosis module 428.The recommendation module 429, in some embodiments, is configured tomake a recommendation based on the diagnosis, with respect to repair,replacement or upgrade.

The device module 430, in some embodiments, is configured to obtaininformation about the device, such as make and model, year ofmanufacturing, last maintenance date, without limitation. Thevisualization module 431, in some embodiments, is configured to displayusage and efficiency scores on a user interface for a user to examine,analyze and understand the data. The annotation module 432, in someembodiments, is configured to allow a user to input annotation, througha user interface, into the database which might be useful for futureanalysis of a device. The comparison module 433, in some embodiments, isconfigured to integrate or compare data from different devices.

FIG. 5 depicts a process flow chart of a method 500 for determining theefficiency (or making a diagnosis) of a device. The various processingoperations and/or data flows depicted in FIG. 5 (and in the otherdrawing figures) are described in greater detail herein. The describedoperations may be accomplished using some or all of the systemcomponents described in detail above and, in some implementations,various operations may be performed in different sequences and variousoperations may be omitted. Additional operations may be performed alongwith some or all of the operations shown in the depicted flow diagrams.One or more operations may be performed simultaneously. Accordingly, theoperations as illustrated (and described in greater detail below) areexemplary by nature and, as such, should not be viewed as limiting.

At step 502, pursuant to a user command, the system, such as personalcomputing device, retrieve or receive energy or resource usage data forthe device or the facility where the device is located for a period oftime. In some embodiments, the period depends on the type of the devicewhich can be, for example, a few months or years for a refrigerator. Atstep 504, the computer system further obtains obtain a property of thefacility and environmental information during the period of time. Withthe property of the facility (e.g., square footage of the facility) andthe environmental information (e.g., outside temperature), the usagedata can be normalized (step 506).

At step 508, the normalized usage data can be analyzed with one or morefeatures useful for making a diagnosis, along with associated dataanalysis model(s) to generate an efficiency score for the device.Evaluation of features can be done with data sets such as by a methodthat calculates a receiver operator characteristics (ROC) curve whereinthe area under curve (AUC) of the ROC curve is indicative of thediagnostic ability of the feature (step 516).

The efficacy so determined can be displayed on a user interface (step512), along with plotting of the original data, or its normalized form(step 510). In addition, suitable recommendations may be made based onthe efficiency results with respect to repair, maintenance, replacementor upgrade (step 514).

FIG. 6 depicts a block diagram of an example computer system 600 inwhich any of the embodiments described herein may be implemented. Thecomputer system 600 includes a bus 602 or other communication mechanismfor communicating information, one or more hardware processors 604coupled with bus 602 for processing information. Hardware processor(s)604 may be, for example, one or more general purpose microprocessors.

The computer system 600 also includes a main memory 606, such as arandom access memory (RAM), cache and/or other dynamic storage devices,coupled to bus 602 for storing information and instructions to beexecuted by processor 604. Main memory 606 also may be used for storingtemporary variables or other intermediate information during executionof instructions to be executed by processor 604. Such instructions, whenstored in storage media accessible to processor 604, render computersystem 600 into a special-purpose machine that is customized to performthe operations specified in the instructions.

The computer system 600 further includes a read only memory (ROM) 608 orother static storage device coupled to bus 602 for storing staticinformation and instructions for processor 604. A storage device 610,such as a magnetic disk, optical disk, or USB thumb drive (Flash drive),etc., is provided and coupled to bus 602 for storing information andinstructions.

The computer system 600 may be coupled via bus 602 to a display 612,such as a cathode ray tube (CRT) or LCD display (or touch screen), fordisplaying information to a computer user. An input device 614,including alphanumeric and other keys, is coupled to bus 602 forcommunicating information and command selections to processor 604.Another type of user input device is cursor control 616, such as amouse, a trackball, or cursor direction keys for communicating directioninformation and command selections to processor 604 and for controllingcursor movement on display 612. This input device typically has twodegrees of freedom in two axes, a first axis (e.g., x) and a second axis(e.g., y), that allows the device to specify positions in a plane. Insome embodiments, the same direction information and command selectionsas cursor control may be implemented via receiving touches on a touchscreen without a cursor.

The computing system 600 may include a user interface module toimplement a GUI that may be stored in a mass storage device asexecutable software codes that are executed by the computing device(s).This and other modules may include, by way of example, components, suchas software components, object-oriented software components, classcomponents and task components, processes, functions, attributes,procedures, subroutines, segments of program code, drivers, firmware,microcode, circuitry, data, databases, data structures, tables, arrays,and variables.

In general, the word “module,” as used herein, refers to logic embodiedin hardware or firmware, or to a collection of software instructions,possibly having entry and exit points, written in a programminglanguage, such as, for example, Java, C or C++. A software module may becompiled and linked into an executable program, installed in a dynamiclink library, or may be written in an interpreted programming languagesuch as, for example, BASIC, Perl, or Python. It will be appreciatedthat software modules may be callable from other modules or fromthemselves, and/or may be invoked in response to detected events orinterrupts. Software modules configured for execution on computingdevices may be provided on a computer readable medium, such as a compactdisc, digital video disc, flash drive, magnetic disc, or any othertangible medium, or as a digital download (and may be originally storedin a compressed or installable format that requires installation,decompression or decryption prior to execution). Such software code maybe stored, partially or fully, on a memory device of the executingcomputing device, for execution by the computing device. Softwareinstructions may be embedded in firmware, such as an EPROM. It will befurther appreciated that hardware modules may be comprised of connectedlogic units, such as gates and flip-flops, and/or may be comprised ofprogrammable units, such as programmable gate arrays or processors. Themodules or computing device functionality described herein arepreferably implemented as software modules, but may be represented inhardware or firmware. Generally, the modules described herein refer tological modules that may be combined with other modules or divided intosub-modules despite their physical organization or storage.

The computer system 600 may implement the techniques described hereinusing customized hard-wired logic, one or more ASICs or FPGAs, firmwareand/or program logic which in combination with the computer systemcauses or programs computer system 600 to be a special-purpose machine.According to one embodiment, the techniques herein are performed bycomputer system 600 in response to processor(s) 604 executing one ormore sequences of one or more instructions contained in main memory 606.Such instructions may be read into main memory 606 from another storagemedium, such as storage device 610. Execution of the sequences ofinstructions contained in main memory 606 causes processor(s) 604 toperform the process steps described herein. In alternative embodiments,hard-wired circuitry may be used in place of or in combination withsoftware instructions.

The term “non-transitory media,” and similar terms, as used hereinrefers to any media that store data and/or instructions that cause amachine to operate in a specific fashion. Such non-transitory media maycomprise non-volatile media and/or volatile media. Non-volatile mediaincludes, for example, optical or magnetic disks, such as storage device610. Volatile media includes dynamic memory, such as main memory 606.Common forms of non-transitory media include, for example, a floppydisk, a flexible disk, hard disk, solid state drive, magnetic tape, orany other magnetic data storage medium, a CD-ROM, any other optical datastorage medium, any physical medium with patterns of holes, a RAM, aPROM, and EPROM, a FLASH-EPROM, NVRAM, any other memory chip orcartridge, and networked versions of the same.

Non-transitory media is distinct from but may be used in conjunctionwith transmission media. Transmission media participates in transferringinformation between non-transitory media. For example, transmissionmedia includes coaxial cables, copper wire and fiber optics, includingthe wires that comprise bus 602. Transmission media can also take theform of acoustic or light waves, such as those generated duringradio-wave and infra-red data communications.

Various forms of media may be involved in carrying one or more sequencesof one or more instructions to processor 604 for execution. For example,the instructions may initially be carried on a magnetic disk or solidstate drive of a remote computer. The remote computer can load theinstructions into its dynamic memory and send the instructions over atelephone line using a modem. A modem local to computer system 600 canreceive the data on the telephone line and use an infra-red transmitterto convert the data to an infra-red signal. An infra-red detector canreceive the data carried in the infra-red signal and appropriatecircuitry can place the data on bus 602. Bus 602 carries the data tomain memory 606, from which processor 604 retrieves and executes theinstructions. The instructions received by main memory 606 may retrievesand executes the instructions. The instructions received by main memory606 may optionally be stored on storage device 610 either before orafter execution by processor 604.

The computer system 600 also includes a communication interface 618coupled to bus 602. Communication interface 618 provides a two-way datacommunication coupling to one or more network links that are connectedto one or more local networks. For example, communication interface 618may be an integrated services digital network (ISDN) card, cable modem,satellite modem, or a modem to provide a data communication connectionto a corresponding type of telephone line. As another example,communication interface 618 may be a local area network (LAN) card toprovide a data communication connection to a compatible LAN (or WANcomponent to communicated with a WAN). Wireless links may also beimplemented. In any such implementation, communication interface 618sends and receives electrical, electromagnetic or optical signals thatcarry digital data streams representing various types of information.

A network link typically provides data communication through one or morenetworks to other data devices. For example, a network link may providea connection through local network to a host computer or to dataequipment operated by an Internet Service Provider (ISP). The ISP inturn provides data communication services through the world wide packetdata communication network now commonly referred to as the “Internet”.Local network and Internet both use electrical, electromagnetic oroptical signals that carry digital data streams. The signals through thevarious networks and the signals on network link and throughcommunication interface 618, which carry the digital data to and fromcomputer system 600, are example forms of transmission media.

The computer system 600 can send messages and receive data, includingprogram code, through the network(s), network link and communicationinterface 618. In the Internet example, a server might transmit arequested code for an application program through the Internet, the ISP,the local network and the communication interface 618.

The received code may be executed by processor 604 as it is received,and/or stored in storage device 610, or other non-volatile storage forlater execution.

Each of the processes, methods, and algorithms described in thepreceding sections may be embodied in, and fully or partially automatedby, code modules executed by one or more computer systems or computerprocessors comprising computer hardware. The processes and algorithmsmay be implemented partially or wholly in application-specificcircuitry.

The various features and processes described above may be usedindependently of one another, or may be combined in various ways. Allpossible combinations and sub-combinations are intended to fall withinthe scope of this disclosure. In addition, certain method or processblocks may be omitted in some implementations. The methods and processesdescribed herein are also not limited to any particular sequence, andthe blocks or states relating thereto can be performed in othersequences that are appropriate. For example, described blocks or statesmay be performed in an order other than that specifically disclosed, ormultiple blocks or states may be combined in a single block or state.The example blocks or states may be performed in serial, in parallel, orin some other manner. Blocks or states may be added to or removed fromthe disclosed example embodiments. The example systems and componentsdescribed herein may be configured differently than described. Forexample, elements may be added to, removed from, or rearranged comparedto the disclosed example embodiments.

Conditional language, such as, among others, “can,” “could,” “might,” or“may,” unless specifically stated otherwise, or otherwise understoodwithin the context as used, is generally intended to convey that certainembodiments include, while other embodiments do not include, certainfeatures, elements and/or steps. Thus, such conditional language is notgenerally intended to imply that features, elements and/or steps are inany way required for one or more embodiments or that one or moreembodiments necessarily include logic for deciding, with or without userinput or prompting, whether these features, elements and/or steps areincluded or are to be performed in any particular embodiment.

Any process descriptions, elements, or blocks in the flow diagramsdescribed herein and/or depicted in the attached figures should beunderstood as potentially representing modules, segments, or portions ofcode which include one or more executable instructions for implementingspecific logical functions or steps in the process. Alternateimplementations are included within the scope of the embodimentsdescribed herein in which elements or functions may be deleted, executedout of order from that shown or discussed, including substantiallyconcurrently or in reverse order, depending on the functionalityinvolved, as would be understood by those skilled in the art.

It should be emphasized that many variations and modifications may bemade to the above-described embodiments, the elements of which are to beunderstood as being among other acceptable examples. All suchmodifications and variations are intended to be included herein withinthe scope of this disclosure. The foregoing description details certainembodiments of the invention. It will be appreciated, however, that nomatter how detailed the foregoing appears in text, the invention can bepracticed in many ways. As is also stated above, it should be noted thatthe use of particular terminology when describing certain features oraspects of the invention should not be taken to imply that theterminology is being re-defined herein to be restricted to including anyspecific characteristics of the features or aspects of the inventionwith which that terminology is associated. The scope of the inventionshould therefore be construed in accordance with the appended claims andany equivalents thereof.

Engines, Components, and Logic

Certain embodiments are described herein as including logic or a numberof components, engines, or mechanisms. Engines may constitute eithersoftware engines (e.g., code embodied on a machine-readable medium) orhardware engines. A “hardware engine” is a tangible unit capable ofperforming certain operations and may be configured or arranged in acertain physical manner. In various example embodiments, one or morecomputer systems (e.g., a standalone computer system, a client computersystem, or a server computer system) or one or more hardware engines ofa computer system (e.g., a processor or a group of processors) may beconfigured by software (e.g., an application or application portion) asa hardware engine that operates to perform certain operations asdescribed herein.

In some embodiments, a hardware engine may be implemented mechanically,electronically, or any suitable combination thereof. For example, ahardware engine may include dedicated circuitry or logic that ispermanently configured to perform certain operations. For example, ahardware engine may be a special-purpose processor, such as aField-Programmable Gate Array (FPGA) or an Application SpecificIntegrated Circuit (ASIC). A hardware engine may also includeprogrammable logic or circuitry that is temporarily configured bysoftware to perform certain operations. For example, a hardware enginemay include software executed by a general-purpose processor or otherprogrammable processor. Once configured by such software, hardwareengines become specific machines (or specific components of a machine)uniquely tailored to perform the configured functions and are no longergeneral-purpose processors. It will be appreciated that the decision toimplement a hardware engine mechanically, in dedicated and permanentlyconfigured circuitry, or in temporarily configured circuitry (e.g.,configured by software) may be driven by cost and time considerations.

Accordingly, the phrase “hardware engine” should be understood toencompass a tangible entity, be that an entity that is physicallyconstructed, permanently configured (e.g., hardwired), or temporarilyconfigured (e.g., programmed) to operate in a certain manner or toperform certain operations described herein. As used herein,“hardware-implemented engine” refers to a hardware engine. Consideringembodiments in which hardware engines are temporarily configured (e.g.,programmed), each of the hardware engines need not be configured orinstantiated at any one instance in time. For example, where a hardwareengine comprises a general-purpose processor configured by software tobecome a special-purpose processor, the general-purpose processor may beconfigured as respectively different special-purpose processors (e.g.,comprising different hardware engines) at different times. Softwareaccordingly configures a particular processor or processors, forexample, to constitute a particular hardware engine at one instance oftime and to constitute a different hardware engine at a differentinstance of time.

Hardware engines can provide information to, and receive informationfrom, other hardware engines. Accordingly, the described hardwareengines may be regarded as being communicatively coupled. Where multiplehardware engines exist contemporaneously, communications may be achievedthrough signal transmission (e.g., over appropriate circuits and buses)between or among two or more of the hardware engines. In embodiments inwhich multiple hardware engines are configured or instantiated atdifferent times, communications between such hardware engines may beachieved, for example, through the storage and retrieval of informationin memory structures to which the multiple hardware engines have access.For example, one hardware engine may perform an operation and store theoutput of that operation in a memory device to which it iscommunicatively coupled. A further hardware engine may then, at a latertime, access the memory device to retrieve and process the storedoutput. Hardware engines may also initiate communications with input oroutput devices, and can operate on a resource (e.g., a collection ofinformation).

The various operations of example methods described herein may beperformed, at least partially, by one or more processors that aretemporarily configured (e.g., by software) or permanently configured toperform the relevant operations. Whether temporarily or permanentlyconfigured, such processors may constitute processor-implemented enginesthat operate to perform one or more operations or functions describedherein. As used herein, “processor-implemented engine” refers to ahardware engine implemented using one or more processors.

Similarly, the methods described herein may be at least partiallyprocessor-implemented, with a particular processor or processors beingan example of hardware. For example, at least some of the operations ofa method may be performed by one or more processors orprocessor-implemented engines. Moreover, the one or more processors mayalso operate to support performance of the relevant operations in a“cloud computing” environment or as a “software as a service” (SaaS).For example, at least some of the operations may be performed by a groupof computers (as examples of machines including processors), with theseoperations being accessible via a network (e.g., the Internet) and viaone or more appropriate interfaces (e.g., an Application ProgramInterface (API)).

The performance of certain of the operations may be distributed amongthe processors, not only residing within a single machine, but deployedacross a number of machines. In some example embodiments, the processorsor processor-implemented engines may be located in a single geographiclocation (e.g., within a home environment, an office environment, or aserver farm). In other example embodiments, the processors orprocessor-implemented engines may be distributed across a number ofgeographic locations.

Language

Throughout this specification, plural instances may implementcomponents, operations, or structures described as a single instance.Although individual operations of one or more methods are illustratedand described as separate operations, one or more of the individualoperations may be performed concurrently, and nothing requires that theoperations be performed in the order illustrated. Structures andfunctionality presented as separate components in example configurationsmay be implemented as a combined structure or component. Similarly,structures and functionality presented as a single component may beimplemented as separate components. These and other variations,modifications, additions, and improvements fall within the scope of thesubject matter herein.

Although an overview of the subject matter has been described withreference to specific example embodiments, various modifications andchanges may be made to these embodiments without departing from thebroader scope of embodiments of the present disclosure. Such embodimentsof the subject matter may be referred to herein, individually orcollectively, by the term “invention” merely for convenience and withoutintending to voluntarily limit the scope of this application to anysingle disclosure or concept if more than one is, in fact, disclosed.

The embodiments illustrated herein are described in sufficient detail toenable those skilled in the art to practice the teachings disclosed.Other embodiments may be used and derived therefrom, such thatstructural and logical substitutions and changes may be made withoutdeparting from the scope of this disclosure. The Detailed Description,therefore, is not to be taken in a limiting sense, and the scope ofvarious embodiments is defined only by the appended claims, along withthe full range of equivalents to which such claims are entitled.

It will be appreciated that an “engine,” “system,” “data store,” and/or“database” may comprise software, hardware, firmware, and/or circuitry.In one example, one or more software programs comprising instructionscapable of being executable by a processor may perform one or more ofthe functions of the engines, data stores, databases, or systemsdescribed herein. In another example, circuitry may perform the same orsimilar functions. Alternative embodiments may comprise more, less, orfunctionally equivalent engines, systems, data stores, or databases, andstill be within the scope of present embodiments. For example, thefunctionality of the various systems, engines, data stores, and/ordatabases may be combined or divided differently.

“Open source” software is defined herein to be source code that allowsdistribution as source code as well as compiled form, with awell-publicized and indexed means of obtaining the source, optionallywith a license that allows modifications and derived works.

The data stores described herein may be any suitable structure (e.g., anactive database, a relational database, a self-referential database, atable, a matrix, an array, a flat file, a documented-oriented storagesystem, a non-relational No-SQL system, and the like), and may becloud-based or otherwise.

As used herein, the term “or” may be construed in either an inclusive orexclusive sense. Moreover, plural instances may be provided forresources, operations, or structures described herein as a singleinstance. Additionally, boundaries between various resources,operations, engines, engines, and data stores are somewhat arbitrary,and particular operations are illustrated in a context of specificillustrative configurations. Other allocations of functionality areenvisioned and may fall within a scope of various embodiments of thepresent disclosure. In general, structures and functionality presentedas separate resources in the example configurations may be implementedas a combined structure or resource. Similarly, structures andfunctionality presented as a single resource may be implemented asseparate resources. These and other variations, modifications,additions, and improvements fall within a scope of embodiments of thepresent disclosure as represented by the appended claims. Thespecification and drawings are, accordingly, to be regarded in anillustrative rather than a restrictive sense.

Conditional language, such as, among others, “can,” “could,” “might,” or“may,” unless specifically stated otherwise, or otherwise understoodwithin the context as used, is generally intended to convey that certainembodiments include, while other embodiments do not include, certainfeatures, elements and/or steps. Thus, such conditional language is notgenerally intended to imply that features, elements and/or steps are inany way required for one or more embodiments or that one or moreembodiments necessarily include logic for deciding, with or without userinput or prompting, whether these features, elements and/or steps areincluded or are to be performed in any particular embodiment.

Although the invention has been described in detail for the purpose ofillustration based on what is currently considered to be the mostpractical and preferred implementations, it is to be understood thatsuch detail is solely for that purpose and that the invention is notlimited to the disclosed implementations, but, on the contrary, isintended to cover modifications and equivalent arrangements that arewithin the spirit and scope of the appended claims. For example, it isto be understood that the present invention contemplates that, to theextent possible, one or more features of any embodiment can be combinedwith one or more features of any other embodiment.

Other implementations, uses and advantages of the invention will beapparent to those skilled in the art from consideration of thespecification and practice of the invention disclosed herein. Thespecification should be considered exemplary only, and the scope of theinvention is accordingly intended to be limited only by the followingclaims.

What is claimed is:
 1. A computer-implemented method for determining anefficiency of a device at a facility, comprising: sensing, by acomputing system, device usage data associated with the device from oneor more sensors; determining, by a machine learning model, one or moreattributes, selected from candidate attributes, that distinguish anefficient operation of the device from an inefficient operation, whereinthe determining of the one or more attributes comprises: obtaining, bythe computing system, data relating to characteristics and environmentassociated with the facility; normalizing, by the computing system, thedevice usage data based on the data relating to characteristics andenvironment associated with the facility; extracting, by the computingsystem, information corresponding to the one or more attributes from thenormalized device usage data at a number of consecutive time intervalswithin the period of time; and determining, by the computing system, aplurality of efficiency scores of the device based on the informationcorresponding to the one or more attributes; and generating, by thecomputing system based on the plurality of efficiency scores, anindication of whether to perform a repair or a maintenance on thedevice.
 2. The computer-implemented method of claim 1, wherein the oneor more attributes comprise a strength of sunlight.
 3. Thecomputer-implemented method of claim 1, wherein the one or moreattributes are selected based on an area under curve (AUC) of a receiveroperator characteristic (ROC) corresponding to a candidate attribute,indicative of a rate of false positives.
 4. The computer-implementedmethod of claim 1, wherein the one or more attributes are selected byselecting a first attribute, wherein the first attribute is a highestperforming attribute among the candidate attributes, and selecting asecond attribute based on a combined performance of the second attributeand the first attribute with respect to an area under curve (AUC) of areceiver operator characteristic (ROC) corresponding to a combination ofthe first attribute and the second attribute.
 5. Thecomputer-implemented method of claim 1, wherein the efficiency scoresare filtered based on an intensity of the device.
 6. Thecomputer-implemented method of claim 1, wherein the indication ofwhether to perform a repair or a maintenance on the device correspondsto a declining relative efficiency of the device prior to a failure ofthe device.
 7. The computer-implemented method of claim 1, furthercomprising: generating, by the computing system based on the pluralityof efficiency scores, an indication of whether to perform an upgrade ora replacement on the device.
 8. A system for determining an efficiencyof a device at a facility, comprising: a processor; and a memory storinginstructions that, when executed by the processor, cause the system toperform: sensing, by a computing system, device usage data associatedwith the device from one or more sensors; determining, by a machinelearning model, one or more attributes, selected from candidateattributes, that distinguish an efficient operation of the device froman inefficient operation, wherein the determining of the one or moreattributes comprises: obtaining, by the computing system, data relatingto characteristics and environment associated with the facility;normalizing, by the computing system, the device usage data based on thedata relating to characteristics and environment associated with thefacility; extracting, by the computing system, information correspondingto the one or more attributes from the normalized device usage data at anumber of consecutive time intervals within the period of time; anddetermining, by the computing system, a plurality of efficiency scoresof the device based on the information corresponding to the one or moreattributes; and generating, by the computing system based on theplurality of efficiency scores, an indication of whether to perform arepair or a maintenance on the device.
 9. The system of claim 8, whereinthe one or more attributes comprise a strength of sunlight.
 10. Thesystem of claim 8, wherein the one or more attributes are selected basedon an area under curve (AUC) of a receiver operator characteristic (ROC)corresponding to a candidate attribute, indicative of a rate of falsepositives.
 11. The system of claim 8, wherein the one or more attributesare selected by selecting a first attribute, wherein the first attributeis a highest performing attribute among the candidate attributes, andselecting a second attribute based on a combined performance of thesecond attribute and the first attribute with respect to an area undercurve (AUC) of a receiver operator characteristic (ROC) corresponding toa combination of the first attribute and the second attribute.
 12. Thesystem of claim 8, wherein the efficiency scores are filtered based onan intensity of the device.
 13. The system of claim 8, wherein theindication of whether to perform a repair or a maintenance on the devicecorresponds to a declining relative efficiency of the device prior to afailure of the device.
 14. The system of claim 8, wherein theinstructions that, when executed by the processor, cause the system toperform: generating, by the computing system based on the plurality ofefficiency scores, an indication of whether to perform an upgrade or areplacement on the device.
 15. A non-transitory computer-readable mediumof a computing system for determining an efficiency of a device at afacility storing instructions that, when executed by a processor, causethe computer system to perform: sensing device usage data associatedwith the device from one or more sensors; determining, by a machinelearning model, one or more attributes, selected from candidateattributes, that distinguish an efficient operation of the device froman inefficient operation, wherein the determining of the one or moreattributes comprises: obtaining data relating to characteristics andenvironment associated with the facility; normalizing the device usagedata based on the data relating to characteristics and environmentassociated with the facility; extracting information corresponding tothe one or more attributes from the normalized device usage data at anumber of consecutive time intervals within the period of time; anddetermining a plurality of efficiency scores of the device based on theinformation corresponding to the one or more attributes; and generating,based on the plurality of efficiency scores, an indication of whether toperform a repair or a maintenance on the device.
 16. The non-transitorycomputer-readable medium of claim 15, wherein the one or more attributesare selected based on an area under curve (AUC) of a receiver operatorcharacteristic (ROC) corresponding to a candidate attribute, indicativeof a rate of false positives.
 17. The non-transitory computer-readablemedium of claim 15, wherein the one or more attributes are selected byselecting a first attribute, wherein the first attribute is a highestperforming attribute among the candidate attributes, and selecting asecond attribute based on a combined performance of the second attributeand the first attribute with respect to an area under curve (AUC) of areceiver operator characteristic (ROC) corresponding to a combination ofthe first attribute and the second attribute comprise metrics selectedfrom the preprocessed training dataset.
 18. The non-transitorycomputer-readable medium of claim 15, wherein the efficiency scores arefiltered based on an intensity of the device.
 19. The non-transitorycomputer-readable medium of claim 15, wherein the indication of whetherto perform a repair or a maintenance on the device corresponds to adeclining relative efficiency of the device prior to a failure of thedevice.
 20. The non-transitory computer-readable medium of claim 15,wherein the instructions that, when executed by the processor, cause thecomputing system to perform: generating, based on the plurality ofefficiency scores, an indication of whether to perform an upgrade or areplacement on the device.